System and method for ingesting and presenting a video with associated linked products and metadata as a unified actionable shopping experience

ABSTRACT

A computer-implemented method for ingesting and presenting a video with associated linked products and metadata as a unified actionable shopping experience may include accessing a video file, receiving information about an object, the information including a link to a merchant portal or web page through which the object may be purchased, combining the video file and information about the object into a social media post, and delivering the social media post to consumers.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This patent application is a continuation of and claims the benefit ofpriority to U.S. application Ser. No. 17/407,338, filed on Aug. 20,2021, now issued as U.S. Pat. No. 11,457,059 on Sep. 27, 2022, whichclaims priority to U.S. Provisional Application No. 63/068,274, theentireties of which are incorporated herein by reference.

TECHNICAL FIELD

Various embodiments of the present disclosure relate generally to onlinepromotion of items in conjunction with social networking and, moreparticularly, to optimizing object placement in online video to enhanceviewer discoverability and actionability of the depicted objects.

BACKGROUND

Businesses create marketing campaign advertisements to market productsto various consumer demographics in the hopes of convincing somepercentage of the consumers to purchase particular products. However,such mass advertisements are costly and relatively ineffective unlessinterested potential consumers receive information regarding the goodsor services in which they are interested. Given the finite budgetsinvolved, it is usually the job function of marketing personnel toobtain sufficient data to assist them in their determination as to whichconsumers to target with the advertisement.

It is commonly understood in business advertising that if it is possibleto accurately determine a potential consumer's desire for or interest ina particular product and provide that consumer with the relevantinformation regarding that particular product when the desire orinterest is greatest, then the chance of the consumer acting on thisdesire or interest and making the purchase is much greater than if theinformation is not received, is incorrect, or arrives at a time wheninterest is low or non-existent. The more certain the desire or interestcan be determined, then the higher the probability of a completed sale.Such an advertising technique is known as targeted marketing, which mayrequire adequate, essentially real-time, consumer information torealize. Thus, actively monitoring a potential consumer's actions,desires, and interests as they occur and develop would be an ideal wayof achieving such information, but such effective data gathering iseffectively non-existent despite current technology advances.

With the rapid evolution of technology there has been a growing trendtoward online publishing by individuals through social media. Popularsocial media networking websites, for example, Instagram, Facebook,Twitter, P interest, YouTube, Snapchat, TikTok, etc. allow users to postuser-generated or acquired images and comments effectively at will. And,because camera-enabled smartphones are ubiquitous, it is relativelysimple for social media users to take and post digital photographs andvideo on these websites and to include commentary. Once posted,subscribers or “friends” of the individual's posts are allowed tocomment on the posts or otherwise rank such posts to indicate the levelof “like” that they share. Such information—ranking by these subscribersor “friends” of particular posts—is invaluable in determining what thesubscriber or “friend” is not only interested in, but also when suchinterest is effectively the greatest. However, efficient access to thisinformation does not exist in current online systems and applications.

One highly relevant exemplar of use of social media networks involvespublishers in the fashion industry, who often share images of favoriteclothing, shoe, and/or accessory fashion items. Such fashion publishersenjoy sharing clothing and accessory “finds” with others through thesesocial media network websites. Upon posting of, for example, aparticularly attractive combination of clothing and accessories,subscribers to the publisher's posts receive notification of such postsand will browse and rank the posted image. In addition to browsing andranking images and videos, it can be desirable to enable users toproceed to shop (i.e., browse and/or purchase) products featured inthose images and videos, by clicking or otherwise selecting formattedURL links to shopping carts and other e-commerce landing pages. Thus,the subscriber demonstrates their interests by shopping products in theapp, or at least records his or her current like or dislike of theclothing and accessories. Near real time, granular access to this datawould allow marketers of the particular clothing and accessories topresent timely relevant advertisements to the subscriber, but suchefficiency has not been possible through conventional methods.

Furthermore, the identification, location, and tagging of promoted itemsin videos is commonly an expensive and time-consuming manual process.This may make the enhancement of such videos to promote the discovery offeatured items difficult. Failure or inability to make such enhancementsmay reduce sales and revenue to the producers and promoters of thevideos.

The present disclosure is directed to overcoming one or more of theseabove-referenced challenges.

SUMMARY OF THE DISCLOSURE

According to certain aspects of the present disclosure, systems andmethods are disclosed for ingesting and presenting a video withassociated linked products and metadata as a unified actionable shoppingexperience.

In one embodiment, a computer-implemented method is disclosed foringesting and presenting a video with associated linked products andmetadata as a unified actionable shopping experience, the methodcomprising: accessing a video file, receiving information about anobject, the information including a link to a merchant portal or webpage through which the object may be purchased, combining the video fileand information about the object into a social media post, anddelivering the social media post to consumers.

In accordance with another embodiment, a system is disclosed foringesting and presenting a video with associated linked products andmetadata as a unified actionable shopping experience, the systemcomprising: a data storage device storing instructions for ingesting andpresenting a video with associated linked products and metadata as aunified actionable shopping experience in an electronic storage medium;and a processor configured to execute the instructions to perform amethod including: accessing a video file, receiving information about anobject, the information including a link to a merchant portal or webpage through which the object may be purchased, combining the video fileand information about the object into a social media post, anddelivering the social media post to consumers.

In accordance with another embodiment, a non-transitory machine-readablemedium storing instructions that, when executed by the a computingsystem, causes the computing system to perform a method for ingestingand presenting a video with associated linked products and metadata as aunified actionable shopping experience, the method including: accessinga video file, receiving information about an object, the informationincluding a link to a merchant portal or web page through which theobject may be purchased, combining the video file and information aboutthe object into a social media post, and delivering the social mediapost to consumers.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1 depicts an exemplary system infrastructure for ingesting andpresenting a video with associated linked products and metadata as aunified actionable shopping experience, according to one or moreembodiments.

FIG. 2 depicts an overall process flow of a system for ingesting andpresenting a video with associated linked products and metadata as aunified actionable shopping experience, according to one or moreembodiments.

FIG. 3 depicts a screen of a publisher user handheld computing devicedepicting an application start screen, according to one or moreembodiments.

FIG. 4 depicts a screen of a publisher user handheld computing devicedepicting a media folder screen, according to one or more embodiments.

FIG. 5 depicts a screen of a publisher user handheld computing devicedepicting a media selection screen, according to one or moreembodiments.

FIG. 6 depicts a screen of a publisher user handheld computing devicedepicting a favorites folders, according to one or more embodiments.

FIG. 7 depicts a screen of a publisher user handheld computing devicedepicting product links available in a particular favorites folder,according to one or more embodiments.

FIG. 8 depicts a screen of a publisher user handheld computing devicedepicting selected favorites consumer product links, according to one ormore embodiments.

FIG. 9 depicts a screen of a publisher user handheld computing devicedepicting entry of a caption for selected media and appended uniqueidentifier, according to one or more embodiments.

FIG. 10 depicts a screen of a publisher user handheld computing devicedepicting a social media network publication selection screen, accordingto one or more embodiments.

FIG. 11 depicts a screen of a publisher user handheld computing devicedepicting a social media network media editing screen and options,according to one or more embodiments.

FIG. 12 depicts a screen of a publisher user handheld computing devicedepicting a social media network media sharing screen, according to oneor more embodiments.

FIG. 13 depicts a screen of a publisher user handheld computing devicedepicting a social media network publisher shared media, according toone or more embodiments.

FIG. 14 depicts a screen of a social media member user handheldcomputing device depicting a social media network screen for member userreview, ranking, and purchase, according to one or more embodiments.

FIG. 15 depicts a portion of a system architecture for ingesting andpresenting a video with associated linked products and metadata as aunified actionable shopping experience, according to one or moreembodiments.

FIG. 16 depicts a portion of a system architecture for deriving objectsfrom a video and mapping them into a content delivery campaign,according to one or more embodiments.

FIG. 17 depicts a portion of a system architecture for identifyingrelative sub-images within an image and determining the effectiveness ofthe placement in content delivery performance, according to one or moreembodiments.

FIG. 18 depicts a flowchart of a method of ingesting and presenting avideo with associated linked products and metadata as a unifiedactionable shopping experience, according to one or more embodiments.

FIG. 19 depicts a flowchart of a method of ingesting and presenting avideo with associated linked products and metadata as a unifiedactionable shopping experience, according to one or more embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS

The terminology used below may be interpreted in its broadest reasonablemanner, even though it is being used in conjunction with a detaileddescription of certain specific examples of the present disclosure.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection.

Any suitable system infrastructure may be put into place to allow forthe hosting and delivery of video related shopping content and infurther deriving object placement in video and optimizing video playbackfor object discovery. FIGS. 1 and 2 and the following discussion providea brief, general description of a suitable computing environment inwhich the present disclosure may be implemented. In one embodiment, anyof the disclosed systems, methods, and/or graphical user interfaces maybe executed by or implemented by a computing system consistent with orsimilar to that depicted in FIGS. 1 and 2 . Although not required,aspects of the present disclosure are described in the context ofcomputer-executable instructions, such as routines executed by a dataprocessing device, e.g., a server computer, wireless device, and/orpersonal computer. Those skilled in the relevant art will appreciatethat aspects of the present disclosure can be practiced with othercommunications, data processing, or computer system configurations,including: Internet appliances, hand-held devices (including personaldigital assistants (“PDAs”)), wearable computers, all manner of cellularor mobile phones (including Voice over IP (“VoIP”) phones), dumbterminals, media players, gaming devices, virtual reality devices,multi-processor systems, microprocessor-based or programmable consumerelectronics, set-top boxes, virtual personal assistants (VPA's), networkPCs, mini-computers, mainframe computers, and the like. Indeed, theterms “computer,” “server,” and the like, are generally usedinterchangeably herein, and refer to any of the above devices andsystems, as well as any data processor.

Aspects of the present disclosure may be embodied in a special purposecomputer and/or data processor that is specifically programmed,configured, and/or constructed to perform one or more of thecomputer-executable instructions explained in detail herein. Whileaspects of the present disclosure, such as certain functions, aredescribed as being performed exclusively on a single device, the presentdisclosure may also be practiced in distributed environments wherefunctions or modules are shared among disparate processing devices,which are linked through a communications network, such as a Local AreaNetwork (“LAN”), Wide Area Network (“WAN”), and/or the Internet.Similarly, techniques presented herein as involving multiple devices maybe implemented in a single device. In a distributed computingenvironment, program modules may be located in both local and/or remotememory storage devices.

Aspects of the present disclosure may be stored and/or distributed onnon-transitory computer-readable media, including magnetically oroptically readable computer discs, hard-wired or preprogrammed chips(e.g., EEPROM semiconductor chips), nanotechnology memory, biologicalmemory, or other data storage media. Alternatively, computer implementedinstructions, data structures, screen displays, and other data underaspects of the present disclosure may be distributed over the Internetand/or over other networks (including wireless networks), on apropagated signal on a propagation medium (e.g., an electromagneticwave(s), a sound wave, etc.) over a period of time, and/or they may beprovided on any analog or digital network (packet switched, circuitswitched, or other scheme).

The system and method is practiced on one or more networked computingdevices. As used herein, the term “automated computing device” or“computing device” or “computing device platform” means a device capableof executing program instructions as streamed or as requested fromattached volatile or non-volatile memory. For example, such a deviceutilizes a microprocessor, microcontroller, or digital signal processorin signal communication with a dedicated and/or shared memory component(RAM, ROM, etc.), one or more network components (NIC, Wi-Fi, Bluetooth,Zigbee, etc.), one or more user input components (keyboard, mouse,touchscreen, etc.), one or more user output or display components,and/or additional peripheral components including a database for bulkdata storage. The computing device may also utilize a standard operatingsystem upon which the program instructions may be executed (OS X, iOS,Linux, UNIX, Android, Windows, etc.) or may utilize a proprietaryoperating system for providing basic input/output. For purposes ofillustration but not limitation, examples of a computing device includemainframe computers, workstation computers, database servers, personalcomputers, laptop computers, notebook computers, tablet computers,smartphones, personal digital assistants (PDAs), or the like, or evensome combination thereof.

As used herein “computer readable medium” means any tangible portable orfixed RAM or ROM device, such as portable flash memory, a CDROM, aDVDROM, embedded RAM or ROM integrated circuit devices, or the like. A“data storage device” or “database device” means a device capable ofpersistent storage of digital data, for example, a computer databaseserver or other computing device operating a relational databasemanagement system (RDBMS), for example, SQL, MySQL, Apache Cassandra, orthe like, or even a flat file system as commonly understood.

As used herein, the term “network” or “computer network” means anydigital communications network that allows computing devices to exchangedata over wired and/or wireless connections, including thetelecommunications infrastructure. Such a network also allows fordistributed processing, for example, through website and databasehosting over multiple computer network connected computing devices. Thepresent invention may utilize one or more such networked computingdevices, with each device physically residing in different remotelocations, including in the “cloud” (i.e., cloud computing). As usedherein, the term “online” means, with respect to a computing device,that the computing device is in computer network communication with oneor more additional computing devices. The term “online” means, withrespect to a user of a computing device, that the user is utilizing thecomputing device to access one or more additional computing devices overa computer network.

As used herein, the term “computer network address” or “network address”means the uniform resource identifier (URI) or unique network address bywhich a networked computer may be accessed by another. The URI may be auniform resource locator (URL), a uniform resource name (URN), or both,as those terms are commonly understood by one of ordinary skill in theinformation technology industry.

As used herein, the term “web browser” means any software applicationfor retrieving, presenting, or traversing digital information over anetwork (e.g., Safari, Firefox, Netscape, Internet Explorer, Chrome, andthe like). A web browser accepts as an input a network address, andprovides a page display of the information available at that networkaddress. Underlying functionality includes the ability to executeprogram code, for example, JavaScript or the like, to allow thecomputing device to interact with a website at a given network addressto which the browser has been “pointed.”

As used herein, the term “digital media” means any form of electronicmedia where media data are stored in digital format. This includes, butis not limited to, video, still images, animations, audio, anycombination thereof, and the like. The term “rank” or “ranking” of thedigital media means the assignment of a value by one viewing the digitalmedia that indicates the one's approval or disapproval, or acceptance,of the digital media. For example, most social media network servicesallow a user viewing a posted digital media to indicate whether theviewer “likes” the posted digital media by allowing the selection of a“like” button (or other such graphical user input device) associatedwith the digital media. The social media network service records andpersists this “like” ranking in a dedicated database as a use metric,associating the ranking with the posted digital media and attributingthe ranking with the member user. Also collected as a metric by socialmedia network services are the digital media views by the member users.Thus, an inference may be made that a user that views a particulardigital media without assigning a “like” ranking either does not likethe digital media or is ambivalent towards the particular digital media.In addition to or alternatively, social media network services may allowa value ranking by, for example, allowing a member user to assign amultiple star value (e.g., 0 to 5 stars) or a number range (e.g., 0 to10), with the higher star count or number range indicating a greater orlesser approval or acceptance of the digital media.

The method steps and computing device interaction described herein areachieved through programming of the computing devices using commonlyknown and understood programming means. For example, stored programsconsisting of electronic instructions compiled from high-level and/orlow-level software code using industry standard compilers andprogramming languages, or may be achieved through use of commonscripting languages and appropriate interpreters. The method stepsoperating on user computing devices may utilize any combination of suchprogramming language and scripting language. For example, compiledlanguages include BASIC, C, C++, C#, Objective-C, Java, .NET, VisualBasic, and the like, while interpreted languages include JavaScript,Perl, PHP, Python, Ruby, VBScript, and the like. For networkcommunications between devices, especially over internet TCP/IPconnections, web browser applications and the like may use any suitablecombination of programming and scripting languages, and may exchangedata using data interchange formats, for example, XML, JSON, SOAP, REST,and the like. One of ordinary skill in the art will understand andappreciate how such programming and scripting languages are utilizedwith regard to creating software code executable on a computing deviceplatform.

FIG. 1 depicts the overall hardware and network architecture aspracticed by a first embodiment of the invention. As shown, the systemutilizes a network connection to a computer network (102), which in thepresent embodiment is a commonly known Internet connection. Alsoenjoying a network connection to the computer network (102) are one ormore social media network services (104). The social media networkservices include any online social media network service that acceptsthe posting of digital media, and allows others to browse the posteddigital media and to apply a ranking to the browsed posted digitalmedia. Examples of social media network services include, but are notlimited to, Instagram, Facebook, Twitter, Pinterest, YouTube, SnapChat,TikTok, and LikeToKnowlt.

The social media network services utilized by the embodiment alsooperate upon automated computing devices providing such functionalitythrough execution of stored program software code and afford third-partyaccess to the service through an application programming interface(API). For example, the Instagram API allows registered third-partyaccess to posted digital media, captions and related metadata (including“likes” of the posted digital media), real-time digital media updates,and other data. One of ordinary skill will readily understand how suchAPI calls may be utilized to obtain the desired data, and will bereadily capable of incorporating such API calls into proprietarysoftware functions designed for such access. Each of the listed examplesocial media network services affords such an API and access to at leastcertain of its data and metadata. In addition to commonly known andpracticed wired (Ethernet or the like) and/or wireless (Wi-Fi and thelike) access to the network (102) also includes cellular networks (106),thereby affording access to the social media networking services usinghandheld computing device cellular phones (112), tablets (118), and thelike.

The system computing devices (108) and data storage devices (110) areconnected to the network (102) to access the social media networkservices (104) using the provided APIs. The system functionalitydescribed herein may be provided on one or more computing devices (108),depending on system requirements. For example, system requirementconsiderations to determine the number and power of computing (108) anddata storage (110) hardware utilized include, without limitation,budgetary constraints, number of actual or anticipated users, failover,redundancy for continuity of service, etc., and are within the skill ofone of ordinary skill.

The system (108) can be accessed by publisher users using wired desktopor personal computing device (114), wirelessly connected computingdevice (116), and/or cellular handheld computing device (112) accessmeans, such means operable to provide network access to the system (108)and social media network services (104) through a web browser. Inanother embodiment a dedicated graphical user interface (GUI) may alsobe provided. Social media network member users can also access thesystem (108) through the same wired (120), wireless (122), and/orcellular handheld device (118) access means. Each publisher user andsocial media network member user computing device is capable ofreceiving program code over the network (102) connection through thedevice web browser or as a download from an application service provideror other such service. Such devices are considered peripheral to thesystem devices (108 and 110).

FIG. 2 is a depiction of the overall process flow of the embodiment,highlighting the high-level interaction among the users, system, andperipheral devices. The diagram is divided to highlight activitiesrelated to the information unique identifier creation device (202)processes and the digital media ranking monitor device andcommunications device (204) processes. The information unique identifiercreation device (206), which is in network communication with the socialmedia network service computing device (104), communicates with apublisher user's handheld computing device (112) as previouslydescribed, following download of the program code as an application froman application provider (for example, the Apple™ App Store) andauthentication of the publisher user's social media network serviceaccount as described in additional detail below. Using the installed andauthenticated application, the publisher user (112) selects a digitalmedia from its device (112) for sharing on the social media networkservice (104) and provides a caption for same. The selected digitalmedia and caption are shared (210) with the information uniqueidentifier creation device (206) of the system, which generates andreturns an information unique identifier in response (212). Thepublisher user then posts this digital media and related informationunique identifier (214) on the social media network service (104) formember users to access and rank. Upon posting, the social media networkservice (104) notifies the system of the posted image and anymodifications thereto (216) for persistent storage of same in the systemdata storage device. Products featured in the posted images or videosmay then be shopped by consumer users of the mobile application, fromdirectly within the mobile application, as described in more detailherein.

A member user's handheld computing device (118) that is also registeredwith the social media network service (104) accesses the posted digitalmedia on the social media network service (104) and provides a “ranking”of the digital media (218). The system digital media ranking monitordevice (208) of the system periodically accesses the social medianetwork service (104) to request a report of the digital media rankings(220), which the service (104) subsequently provides (222). Thisreceived ranking and digital media information is then persisted in thedata storage device (110) for subsequent storage and processing. Thecommunications device (208) of the system periodically sends targetedmarketing information to the member user that is relevant to the memberuser's digital media rankings (228) and purchases.

FIG. 3 is a depiction of the screen of a publisher user handheldcomputing device (300) depicting the application start screen asprovided by an embodiment. Shown is the start screen of the downloadedhandheld computing device application portion of the system. Oncedownloaded, the application icon (302) appears on the publisher user'sscreen. Upon selecting the icon to start the application, the publisheruser is presented with the system start screen.

In accessing the system functionality, the publisher user must firstselect a digital media file for posting on the social media networkservice. FIG. 4 is a depiction of the screen of a publisher userhandheld computing device (300) depicting the media folder screen asprovided by the embodiment. The publisher user is presented with foldersfrom which to select the digital media file for posting (402). Uponselection of a folder, the digital media files are made available. FIG.5 is a depiction of the screen of a publisher user handheld computingdevice depicting the media selection screen as provided by theembodiment. The publisher user selects a digital media file for use(502). Once selected, the publisher user is presented with his or herfavorites folders as maintained by the system.

FIG. 6 is a depiction of the screen of a publisher user handheldcomputing device depicting the favorites folders as provided by theembodiment. As stated above, the favorites folders in this embodimentcontain favored consumer product image data/metadata and links tosources of various consumer items of interest to the publisher user. Thepublisher user selects a favorites folder (602) containing consumeritems visible in the selected digital media file, and is presented witha display of the favored consumer products stored therein as depicted inFIG. 7 .

FIG. 7 is a depiction of the screen of a publisher user handheldcomputing device (300) depicting the product links available in aparticular favorites folder as provided by the embodiment. Shown in thefigure are favored consumer products retained by the system in thepublisher user's selected favorites folder (702). Publisher users alsohave the ability to search the contents of the folder (704) in the eventof a large number of stored items, or of the system database for othersuch saved consumer product image data/metadata. In addition tosearching, browsing, and ranking images and videos, the products linksdescribed above enable users to shop (i.e., purchase) products featuredin those images and videos, by clicking or otherwise selecting formattedURL links to shopping carts and other e-commerce landing pages. FIG. 8is a depiction of the screen of a publisher user handheld computingdevice depicting the selected favorites consumer product links asprovided by the embodiment. Depicted are the selected products (802) inthe folder that the publisher user wishes to highlight in the chosendigital media file (804). In this case, the digital media is aphotographic image of a female's wrist with stacking bracelets (804).The images from the favorites folder that were chosen (802) are thebracelets in the image (804).

FIG. 9 is a depiction of the screen of a publisher user handheldcomputing device (300) depicting the entry of a caption for the selectedmedia and appended unique identifier as provided by the embodiment.After selecting the items from the favorites folder or from the resultsof a search of the system database, the publisher user is presented witha screen for entry of a caption to accompany the digital media file(902), preferably a description of the image contents. Below the captionentry is a listing of the selected items, including item metadata (forexample, purchase price, commission amounts, source for item, etc.).

Following selection of items and entry of a caption, the publisher useris presented with the pre-publication screen for publication to thesocial media network service. FIG. 10 is a depiction of the screen of apublisher user handheld computing device (300) depicting the socialmedia network publication selection screen as provided by theembodiment. In this instance, the digital media and data are intendedfor publication on the Instagram social media network service, which maybe accomplished by selecting the Instagram icon (1002). It is alsopossible for the publisher user to copy the information uniqueidentifier that is presented to a web browser clipboard or text editor,and paste the copied information unique identifier as a caption orcomment manually in a social media posting.

Upon selection of the Instagram social media network service icon(1002), the publisher user interface changes to the social media networkservice image modification screen. FIG. 11 is a depiction of the screenof a publisher user handheld computing device (300) depicting the socialmedia network media editing screen and options as provided by theembodiment. As shown, the publisher user is provided with numerousfilters (1102) that may be applied to the digital media image, includingrotation, brightness, and contrast adjustments (1104). Once the digitalmedia image is adjusted, if desired, the image is ready for posting onthe social media network website.

FIG. 12 is a depiction of the screen of a publisher user handheldcomputing device depicting the social media network media sharing screenas provided by the embodiment. This screen presents the publisher userwith the ability to edit the digital media file metadata (1202) and topost the posted image on a selection of other social media networkservices (1204). Once formally published, the publisher user ispresented with the posted digital media file as it appears on the socialmedia network service website as in FIG. 13 . FIG. 13 is a depiction ofthe screen of a publisher user handheld computing device depicting thesocial media network publisher shared media as provided by theembodiment. Visible on this screen is the posted digital media file(1302), the caption (1304) containing the information unique identifierelements (1306), and the ranking feature (1308). The digital media fileis now formally posted on the social median network service website andthe system is notified and the information unique identifier isactivated by the system.

FIG. 14 is a depiction of the screen of a social media member userhandheld computing device (1400) depicting the social media networkscreen for member user review, ranking (e.g., “liking,” “hearting,” or“favoriting,” and purchase (i.e. for the products to be “shopped”) asprovided by the embodiment. In this example, the digital media fileimage (1402) is presented as posted, with the caption containing theinformation bundle presented below (1404). The member user is given theoption to rank the image by selecting the “like” button (1406), and/ormay add a comment (1408) to the post. In another embodiment the postedcomment is retrieved by the system in the same fashion as the ranking,and is utilized alone or in conjunction with the ranking to furtherrefine the targeting of marketing information to the member user.Finally, and most desirably, the user may be enabled to select aformatted URL link, icon, button, or other user element to shop productsfeatured in the media. In one embodiment, the shopping URL link, icon,button, or other user element may be embedded within the media (e.g.,within a photo or video). In another embodiment, the shopping URL link,icon, button, or other user element may be positioned below or adjacentto the media. The shopping URL, link, icon, button, or other userelement may call a pop-up, a preview of a product, a shopping cart, orany other e-commerce interface enabling the user to shop productsfeatured in the media.

Ingesting and Presenting a Video with Associated Linked Products andMetadata as a Unified Actionable Shopping Experience

A content author, often referred to as an influencer or creator, maymake use of the systems and user applications described above to providecontent to other users, sometimes referred to as “followers” of theinfluencer or creator. This content can be, but is not limited to, videocontent, products in the form links and image assets, as well as othermetadata such as descriptions, hash-tags, titles, author handles, etc.An instance of this content, in aggregate form, is generally referred toas a post.

User applications, such as those described above, may provide a userexperience allowing the creator to select a video from local storage,associate a number of products with the video, and add supportingmetadata such as, for example, a post description. These assets takentogether constitute an enriched or enhanced post that is more engagingand shoppable.

Selecting a video from local storage may include, but is not limited to,selecting that video from an existing camera roll platform feature orwidget. When a video is selected for inclusion into a post, a processmay be presented by which trimming and other edits can be performed tothe video. These edits can be performed in both a destructive ornon-destructive manner, either preserving or overlaying the originalvideo content on the local device.

The selection of products may be a result of pre-existing product linksbeing created in a separate product link generation process. Theseproduct links can be both created and favorited by the creator inseparate processes. Favorited products are made available to the creatorduring this post creation process and may then be further associatedwith the current post. The linked products may be presented in variousforms other than those marked as favorites or provided generally as afavorite. Any number of products may be included in a post or a limitmay be imposed in either client or server side logic.

When a creator is satisfied with the constructed post, the creator maythen submit that post to be shared with a consumer audience. During thisprocess, the video and other associated assets related to the post maybe sent to a server process. This server process may process the postmaterials for display to end users.

In the case of video, the video file may be uploaded from the clientdevice and may be further directed by associated server processes to berouted through a transcoding process. This transcoding process ismulti-faceted in that there may be several outcomes associated with theoverall process. Primarily, the transcoding process is meant to convertthe incoming video file into one or more files, all of which areconsumable by various mobile platform devices, including, for example,iOS and Android, etc. This may ensure that video produced on one devicewill be compatible with the playback capabilities of another device.Additionally, the transcoding operation may result in creation ofseveral files, each with their own resolution and bitrate. This mayallow for the delivery of video content to a mobile device encoded at abitrate which is compatible with the current bandwidth of the device,taking network conditions into consideration, and may allow for aquality, buffer-minimizing user experience. In addition, an associatedplaylist file may also be created such as in the format of an m3u8playlist. The files resulting from the transcode step may be containedwithin this playlist. By presenting this playlist file to the consumerfacing application for playback, such as by a URL, the device platform,generally iOS or Android, may select the most representative file URLwithin the playlist to be sent to the client device and streamed ordownloaded, again possibly taking into consideration network conditions.The processing of the other associated post assets may be handled asappropriate to the type of post asset.

When the processing of the post materials is complete, a message may besent to the creator, such as by way of the creator's mobile device,indicating that the post is available for viewing by a consumer. Inaddition, the post may be categorized by the creator to be published ona date and time, of their choosing, in the future. When a consumerexperiences a post, they may experience it as part of a feed of posts,delivered in either a continuously scrollable or paginated fashion. Thisfeed information may be delivered to a consumer device via anapplication programming interface (API). This API may be processed bythe consumer facing application. Upon the completion of the API on theconsumer device, information relating to a collection of posts may bedelivered in a machine or human readable format such as Java scriptobject notation (JSON), or other suitable format. This consumerexperience may comprise, for example, a video, associated products, inthe form of images with associated links and other associated metadata.The product images may be set to be clickable and may allow for theprocessing and routing of a clicked product to take a consumer throughthe retailer or brand's variant selection process, in terms of selectingsize or color variants, and ultimately completing the buying process fora product. Additionally the transcoding and processing steps outlinedabove may be subject to other more advanced features includingobject/product recognition, branded logo insertion, etc.

These features may be provided in a method for ingesting and presentinga video with associated linked products and metadata as a unifiedactionable shopping experience, such as is shown in FIG. 19 . As shownin FIG. 19 , at operation 1910, the method may access a video file. Atoperation 1920, the method may receive information about an object, theinformation including a link to a merchant portal or web page throughwhich the object may be purchased. At operation 1930, the method maycombine the video file and information about the object into a socialmedia post. At operation 1940, the method may deliver the social mediapost to consumers.

Enabling Sharing of Videos and Streaming Media

The platform discussed above may provide for sharing video-type content,as well as additional features relating to sharing of videos ofproducts, whether pre-recorded, live video, streaming or otherwise. Forexample, the platform may be used to provide an interface for uploadingvideos of products consistent with the present disclosure. Thesefeatures may enable users of the platform (e.g., “influencers”) toproduce, monetize and share short (e.g., up to two minute, five minute,etc.) videos in the shopping app (e.g., rewardStyle's LIKEtoKNOW.itapp), including beauty how-tos, home tours, expanded outfit-sharing andproduct reviews. Thus, all of the features described herein may becombined with techniques for enabling users of the application to shop(e.g., browse and/or purchase) products featured in the media, whetherphotographic, recorded video, or live/streaming video. Shopping videoperformance data and analytics may also be available immediately in anassociated influencer marketing management dashboard, where brands canmanage proprietary performance information like insights into RevenueProducing Influencers (RPI), channel performance, app statistics andcampaigns reporting.

Additional interfaces may be provided to select and add media, trim themedia, and add captions and products to the uploaded media (e.g.,recorded or live streaming video).

Other interfaces may be provided to select a thumbnail image, and addhashtags or other labels to the video content. The posted media, whetherphotographic, recorded video, or live/streaming video may then beshopped within the mobile application in which it appeared. As describedabove, the products may be shopped based on a user interacting with themedia itself, user icons in or around the media, or other links or iconswithin the mobile application that call one or more desired e-commerceinterfaces.

As described above, one or more embodiments may include additionalfeatures that are layered onto an influencer or photo/video-sharingplatform (e.g., mobile app). In other words, the above-describedsystems, interfaces, etc. may all be used in combination with any of thefollowing described features.

Creating Custom Branding Logos in Shareable Content

Influencers often wish to have their own custom logo on their videos.The also wish to have the ability to change their logos as they see fitover time.

When a user chooses to change their visual branding, the process maybegin by creating a large asset for the largest display that may besupported for the given media type. This may be filtered down to ageneric iterating task to iterates through a complete list of media tobe rebranded. One or more embodiments may use scalable vector graphics(SVGs), to support them changing a username (should the user so chooseto have their username in their brand image). By ensuring the SVG usinga high fidelity typeset, one or more embodiments may ensure that agenerated asset may scale to lower dimensions for display on lower sizevideos.

However, most implementations of media transcoders do not properly scaleimages such that they preserve fidelity and smooth curves; rather theycreate choppy artifacts. Accordingly, one or more embodiments maypre-scale images before handing the images off to a transcode task thatadds the overlay to the media. This transcode process may use theoriginal uploaded source media.

A data store may also be utilized to track when a given client has arebrand activity in progress and to keep track of important attributeslike task start time, input configuration(s), and task completion time.

One or more embodiments may ensure that assets are not recreated for agiven media's dimensions through use of a cache system such that theonly context the task iterator needs to track are the sets of inputmedia and the brand overlay definition. This may allow the processfunction at scale.

Deriving Multi-Lingual Search Engine Optimization from Video

An influencer-based marketing platform according to one or moreembodiments may have the ability to host both video and still images.These videos and images are generally referred to as “hero” shots, andmay be accompanied by caption text, hashtags and associated productlinks that relate to the video or still images. In the specific case ofvideo, video is often accompanied by an audio track which also includesspoken language. During a complex transcoding function which takes avideo file as an input stream and outputs a different type of file,optimized for streaming or mobile consumption, the audio track may beisolated and routed to a further transcoding operation. This transcodingoperation may route the audio track through a speech recognition processwhich may create a time stamped text file in an XML or other format.This file may then be further processed by a text translation processrunning in multiple different language contexts. The output of thisprocess may be a number of files in XML or other formats. These outputfiles may then be input to search engine optimization components and mayalso be used within a conversational computing environment which mayincrease the efficiency of finding materials relating to a specificsource video. Such a process may also relate to still images as theseimages may include caption text and other file specific taggingartifacts.

Ensuring Branded Overlay Clarity on Low Resolution Video

A user providing a relatively low pixel count or low bitrate video maydesire a branded video that clearly shows their brand, regardless of theinput stream quality.

For a given video with a given number of pixels, there may exist someratio of video bitrate per pixel per pixel depth that provides anaesthetically pleasing display of a brand logo where artifacts gained inthe logo during a transcode process are minimized. If a video were totranscoded below this bitrate per pixel per pixel depth threshold thebrand image may be at risk of turning fuzzy or gaining artifacts notappealing to the end user. One or more embodiments may includeexperimentation for arbitrary video output codecs such that an optimalvalue may be derived for that codec and ensuring each video goingthrough a branding process according to one or more embodiments uses abitrate per pixel per pixel depth that meets or exceeds this value.

Such an approach may be customized per brand partner if their brandingis more dynamic than a single image. This may be achieved implicitly byoptimizing for known output formats if using static media such asimages. In addition a trained image recognition algorithm may be used onhigh entropy frame transitions to ensure the brand image may berecognized if it is a static picture.

Publishing A Streamable Media With Increasing Bandwidth Modalities

When a client uploads a video media file with the intent to share andpublish to a following audience, the client often wants to minimize thetime to first stream such that they can minimize their time spentwaiting on backend services.

A traditional way of transcoding can be to read a source stream once andcreate N output streams for the various streaming bitrates and purposesthat are desired. This approach is often an 10 bound operation, meaningthe more outputs created in a single transcode operation, the longer theoperation takes to complete. To minimize wait time, one or moreembodiments may declare a system of identities and references such thata general streaming media source, such as an HTTP Live Streaming (HLS)stream identity, gains additional maximum bitrates over time.

According to one or more embodiments, a given media stream may be aguide that maps to an HLS .m3u8 file via a record in a datastore. Ondisplay of a context wrapping an identifier of a media, the currentlocation of a processed m3u8 file in a content delivery network (CDN)may be returned. If the input file is already in a format optimized forHLS streaming, an HLS stream .m3u8 file may be retroactively created forthe uploaded video file and both the original video file and streamdefinition to a CDN origin may be published. If transcoding of theuploaded video file operation is needed, a subset of video bitrates maybe chosen such that low bandwidth and average bandwidth clients mayreceive an optimized experience and the associated HLS .m3u8 files maybe published to a CDN origin along with their underlying video chunkfiles. This may reduce the time the client is waiting for the mediastream to be available for publication.

After these operations, an asynchronous job may be constructed totranscode the remaining stream bitrates and merge these chunks into anew HLS stream, such as by referencing the already existing streamswhere possible. All the original stream files may be left as they wereand may not be deleted until a later time through another asynchronousjob, possibly with special consideration to push redirect rules to theCDN for any previous .m3u8 file that was removed.

All HLS streams and CDN locations may be stored under a transcode jobguide directory path so that a service in front of the data store mayremap a media stream ID to a stream that supports more bandwidthmodalities as they become available.

Special consideration may be taken to not run multiple transcodes thatwould merge stream files at the same time. Further, as stream locationsare updated, clients may be informed of the additional modalitiesavailable should the client have a less than optimal viewing experiencefor their available bandwidth.

Processing Asynchronous Callbacks Regardless of Client Application State

One or more embodiments relate to the coordination of messaging betweena client and the associated backend services it relies on. Thisevent-based system may coordinate, prioritize, and associate backgroundprocesses and may queue with specific notifications exposed on theclient side. In general, server-side processes may complete tasks in anasynchronous manner, the methods according to one or more embodimentsmay create an orchestration component which may use sophisticated logicto rationalize state data. This state data may then be processed fordisplay in a human facing application and may deliver the flow of stateinformation that the human needs in order to work efficiently within anapplication context.

Providing In-Store Navigation to Favorited Products

Through a mobile app, according to one or more embodiments, influencersmay post still images and videos complemented by caption text, hashtagsand products. These products may be bought online through the app or mayalso be purchased at a physical store location. One or more embodimentsmay provide product data in combination with integrations into retailerproduct inventory databases. Further one or more embodiments mayinterface with indoor retailer beacon-based geolocation systems todirect individuals who have favorited a product, shown interest in aproduct, or who's profile matches specific products to in-storeinventory and geolocation. One or more embodiments may extend towearable smart watches and operate either dependent or independent ofconnections between phone platforms and wearable devices.

Video Layout Determination in Dependence on Adjacent Visual Assets

One or more embodiments may create specific asset positioning, croppingand sizing relating to images, videos, widgets or containers. One ormore embodiments may relate to mobile and web computing and renderingenvironments. One or more embodiments may also provide contextualfeatures which may affect the rendering of one object in dependence onadjacent objects. One or more embodiments may further make use ofspecific metadata tagging of objects and techniques relating to imagerecognition and analysis that may take place in either real-time oroffline batch related processes.

System and Method for Orchestrating Audio Seamlessly Across MultipleApplication Forms

One or more embodiments may provide a system wide, application wide,seamless orchestration of audio playback with a complex mobileapplication. One or more embodiments may be based upon a global featurewhich may coordinate the playback of audio content within a multi-formapplication and may ensure that audio playback is aligned to thespecific user experience currently in focus. One or more embodiments mayalso allow for background audio playback, independent of videorendering.

System and Method for Orchestrating Video Seamlessly Across MultipleApplication Forms

One or more embodiments may provide a system wide, application wide,seamless orchestration of video playback with a complex mobileapplication. One or more embodiments may be based upon a global featurewhich may coordinate the playback of video content within a multi-formapplication and may ensure that audio playback is aligned to thespecific user experience currently in focus. One or more embodiments mayalso allow for background audio playback, independent of videorendering.

Deriving Product Placement in Videos and Optimizing Video Playback forProduct Discovery

In general, sales-oriented videos contain a number of images containedwithin the frames of the video. The placement, angle or poses of aproduct within a video may affect the emotional connection that is madeto a viewer of the video. One or more embodiments may identify objectsin a video frame from a large library of images of objects andinformation about those objects. By measuring placement, angle, screentime, lighting and many other factors, one or more embodiments mayderive quality coefficients related to how a viewer perceives thatobject in the video. By correlating the derived coefficients to viewerengagement with the depicted object, such as through sales numbersassociated with linked products and objects associated with the video,one or more embodiments may further derive overall performance metrics.These performance metrics may then be used in the analysis of acollection of videos that further datamine metrics. These metrics maythen be applied as guidance when promoters of the depicted object, suchas social media influencers, link products similar to the depictedobjects to their posts. The information derived from the videos andviewer engagement may guide promoters in creating future videos thatobjects to be promoted.

Thus, promoters, such as social media influencers, increasingly referredto as creators, may benefit from such a system to analyze objectplacement in videos and provide guidance to enhance the performance ofsuch videos. FIG. 15 depicts a portion of such a system architecture forderiving object placement in video and optimizing video playback forobject discovery, according to one or more embodiments.

As shown in FIG. 15 , a method for deriving object placement in videoand optimizing video playback for object discovery, such as may beimplemented by frame analysis module 1520, may initially access a videofile from a database, such as media repository 1510. Frame analysismodule 1520 may analyze the video file such as to determine acorrelation between individual frames of the video file and a set oflinked products associated with a publication, such as a social mediapost. The analysis may make use of image recognition in the video frame,such as by image recognition algorithm 1530. For example, imagerecognition algorithm 1530 may first isolate individual objects withinthe video frame, and then may extract the coordinates where eachisolated object is placed within the video frame. Image recognitionalgorithm 1530 may then, at operation 1540, write object coordinateinformation to a database, such as analysis data store 1550, as, forexample, database records, metadata or other recordings. The objectcoordinate information may include multiple positions within a set ofvideo frames to reflect movement of an object with respect to multipleconcurrent video frames. Other information related to the objects in thevideo frames, such as identification information about the objects, mayalso be recorded in analysis data store 1550. Through a comparison ofobject information derived at various scales, from multiple videos, suchas a varying number of video files or other content, inferences may bederived and recorded. For example, in addition to using artificialintelligence methods and other image recognition techniques to ascertainwhich products are associated with a video, further comparisons may bemade between unrelated videos to determine if the same, or a subset ofproducts, are contained in each video. These measurements may allow forselective presentation to a consumer of multiple videos, or othercontent, found to have related products.

Comparing multiple videos, possibly on the scale of thousands ormillions of videos, may yield results, and therefore, insightssupporting decisions regarding the products within these videos, who (interms of what influencer or creator) posted the video, how well thesevideos performed in promoting certain products, the return onadvertising spend (ROAS) produced as a result of the video, and thelikelihood of a certain influencer or influencers being well suited toparticipating in paid campaigns for a brand or retailer. Additionally,comparisons may be multifaceted across several different metrics. Thesemetrics may include, for example, when in the playback of a video doesthe user interact with the post (e.g., like the post, share the post,inspect the author, follow the author, inspect product details, add aproduct to a wishlist, favorite a product, browse product variations,search product retailers, enable audio, disable audio, pause playback,resume playback, seek a specific playback point of a video) or passivelyinteract with the post (e.g., how many times the user watched the video,the time spent viewing the post before leaving this post's context)along with user attributes (audio state enabled/disabled, deviceplatform, device form factor, device OS version, user's locality/timezone, user's gender, user's demographic or other data). Further, giventhe metrics captured combined with the test variations applied, togetherwith a set of interaction-based goals set by the creator, brand orretailer, campaign manager, or others, a test strategy may be determinedyielding desired results and by what margins those results are comparedto other strategies. In addition, this captured data may be used torecommend other strategies for video content generation which may or maynot have been considered previously, and which may yield significantgains in user engagement.

The derived inferences may result in guidance given to a social mediaauthor contributing the video, or may be generalized and given asguidance to multiple social media authors. This guidance may be curatedand delivered as, for example, emails, push notifications, in-appmessaging or other forms of distribution.

The social media author providing the video, or another entity involvedin the promotion or marketing of the objects depicted in the video, maywant to use the video to bring the object or product to the attention ofothers who may have interest in the object, such as potential purchasersof the object. An additional process may use the coordinate data derivedfrom the image recognition algorithm and create metadata for the video,possibly in frame-wise fashion, to map a uniform resource locator (URL)or other link to a product, thus potentially allowing a user obtainfurther information about the object, such as options for purchasing, byinteracting with the object within the video display, such as by simplytouching, clicking, or selecting the object or product shown in thevideo. Further decoration to that object may optionally be incorporatedinto the video or on playback using a related file of events correlatedto the video. These decorations may be, for example, bounding boxes,glitter effects, highlighting effects, etc.

The videos provided by social media authors may be used by those authorsand by brands, retailers, or manufacturers to illustrate and promoteproducts. For example, purchases or other interactions with the productslinked from object images in the video may generate commissions for thesocial media author. FIG. 16 depicts a portion of a system architecturefor deriving objects from a video and mapping them into a contentdelivery campaign, according to one or more embodiments. Theframe-oriented image recognition method shown in FIG. 16 may be used toidentify specific objects or products depicted in the fames of a video.The illustrated process may identify objects from a video or still imagesource file, recognize that image and use metrics associated with theobjects to optimize user interaction with the objects, such as byoptimizing a retail campaign. Identified products may then be stored ina database in either structured or unstructured forms. When brands andretailers want to construct a campaign, the process may match theobjects, items, or products for the campaign with items found in thesource videos and correlated to coupled products. This may match certainexisting videos with the objects, items, or products to be featured. Thesystem may then map an object, item, or product to be featured in postsby specific social media authors and may automatically assemble acampaign with reduced human guidance or intervention.

As shown in FIG. 16 , a method for deriving objects from a video andmapping them into a content delivery campaign, may initially access avideo file from a database, such as media repository 1510. In operation1605, the method may identify the video based on information and/ormetadata in a publication, such as data 1610 of a social media post madeby a social media author. In operation 1615, the method may derivecertain metadata, such as information about object, items, or productsfeatures in the social media post or the video, based on data 1610. Inoperation 1620-1625, the method may model one or more libraries from thederived metadata. For example, more than one model library may beproduced from metadata for the purpose of using an artificialintelligence method, such as a machine learning algorithm, to refinetraining for the subsequent processing of videos using improved imagerecognition. In one or more embodiments, the method may present to amachine learning process, one or more product, or object, libraries.These libraries may be used by a machine learning process. When productsare used in the context of a post, and these products do not havesufficient metadata to fully relate the product to a product category,model libraries, used with specific machine learning algorithms, allowthese associated algorithms to identify the products, associate with aproduct category, resolve to a particular product variant (e.g. color,size, etc).

In operation 1630, the method may extract one or more individual framesfrom the video. Image recognition algorithm 1530 may first isolateindividual objects within the extracted video frames based on theinformation from operation 1620, and then may extract the coordinateswhere each isolated object is placed within the video frame. Imagerecognition algorithm 1530 may then, at operation 1540, write objectcoordinate information to a database, such as analysis data store 1550,as, for example, database records, metadata or other recordings. Theobject coordinate information may include multiple positions within aset of video frames to reflect movement of an object with respect tomultiple consecutive video frames. Other information related to theobjects in the video frames, such as identification information aboutthe objects, may also be recorded in analysis data store 1550.

In operation 1650, the method may allow the platform applicationprovider or additionally a brand or retailer to access product andproduct placement data. This may further allow for the additionalcorrelation between an intentional or “paid” campaign, initiated betweenthe platform application provider and the brand, to verify productplacement within the post, and further measure the effectiveness of thatplacement.

In operation 1645, the method may allow for a number of algorithmicprocesses to gather and analyze the metrics produced and stored inanalysis data store 1550.

In operation 1640, the method may provide a method for correlating theperformance of posts, generally but not limited to a calculation ofreturn on advertiser spend (ROAS), and further associate these postperformance measurements with the posting creator or influencer

In operation 1635, the method may use the algorithmically derivedconclusions relating to post performance as an input to an intelligentcasting process which maximizes the likelihood of a particularinfluencer having promotional success with a particular product.

Further insights into the presentation of items in a video may be gainedthrough an analysis of the placement of items in multiple videos, socialmedia posts, and the performance of campaigns associated with thoseitems. Such analysis may include determining a positional relationshipof items in video frames, storing this position metadata, anddetermining a geometric relationship between multiple frames. The framesmay then be individually processed through additional image recognitionprocesses to categorize each item, such as in terms of product type.This image recognition data may then be compared with campaignperformance data for other videos and social media posts to determinehow similar or different campaigns have performed based on thepositioning of items in a group image or based on the relationalposition in other images. This analytics information may then be used tobuild dynamic guidance shared with social media authors to relate how tocreate better optimized group images or collages. This method may alsosynthetically create optimized collages for either automatic or humanreviewable content. This content may ultimately be used in the contextof social media posts, but is not limited to use in that context.

FIG. 17 depicts a portion of a system architecture for identifyingrelative sub-images within an image and determining the effectiveness ofthe placement in content delivery performance, according to one or moreembodiments. The illustrated process may identify items, objects, orproducts from a video or still image source file, recognize individualitems or other artifacts within the image and use the metrics associatedwith the analysis of the positional product placement to determine theperformance of an item at the positional level and further use that datato optimize a campaign to feature or promote that item.

As shown in FIG. 17 , a method for deriving objects from a video andmapping them into a content delivery campaign, may initially access avideo file from a database, such as media repository 1510. In operation1605, the method may identify the video based on information and/ormetadata in a publication, such as data 1610 of a social media post madeby a social media author. In operation 1615, the method may derivecertain metadata, such as information about object, items, or productsfeatures in the social media post or the video, based on data 1610. Inoperation 1620-1625, the method may model one or more libraries from thederived metadata. For example, more than one model library may beproduced from metadata for the purpose of using an artificialintelligence method, such as a machine learning algorithm, to refinetraining for the subsequent processing of videos using improved imagerecognition. In one or more embodiments, the method may present to amachine learning process, one or more product, or object, libraries.These libraries may be used by a machine learning process. When productsare used in the context of a post, and these products do not havesufficient metadata to fully relate the product to a product category,model libraries, used with specific machine learning algorithms, allowthese associated algorithms to identify the products, associate with aproduct category, resolve to a particular product variant (e.g. color,size, etc).

In operation 1630, the method may extract one or more individual framesfrom the video. Image recognition algorithm 1530 may first isolateindividual objects within the extracted video frames based on theinformation from operation 1620, and then may extract the coordinateswhere each isolated object is placed within the video frame. Imagerecognition algorithm 1530 may then, at operation 1540, write objectcoordinate information to a database, such as analysis data store 1550,as, for example, database records, metadata or other recordings. Theobject coordinate information may include multiple positions within aset of video frames to reflect movement of an object with respect tomultiple concurrent video frames. Other information related to theobjects in the video frames, such as identification information aboutthe objects, may also be recorded in analysis data store 1550.

In operation 1645, the method may allow for a number of algorithmicprocesses to gather and analyze the metrics produced and stored inanalysis data store 1550. Such metrics may include, for example,correlations in post (Hero shots, Products and Metadata) performancesuch as, but not limited to ROAS, products contained within the post,products contained with the video frames, etc.

In operation 1710, the method may provide a correlation to a video orframe within a video related to a set of coordinates which may furtherrelate to a bounding area within the video or video frame where anobject or product has been located. Operation 1710 may further refinethe correlations identified by operation 1645, such as by establishingthe relationship to a product's placement within a video. Additionalmeasurements may include the length or number of frames in which aproduct is rendered in a video, the angle of the product in the video,whether other objects are in the same frame or frames of the video, etc.

FIG. 18 depicts a flowchart of a method of deriving object placement invideo and optimizing video playback for object discovery, according toone or more embodiments. As shown in FIG. 18 , in operation 1805, themethod may access a video file. The video file may be accessed from adatabase, such as media repository 1510 depicted in FIG. 15 . Inoperation 1810, the method may receive information about a set ofobjects. The objects may be, for example, items that a social mediapromoter wishes to feature or promote in a social media post, productsthat a manufacturer, retailer, or brand wishes to advertise, or anyother type of object. In other circumstances, the object may berepresentative of a service provided by the social media author oranother entity, that the social media author or other entity wishes topromote. The information about the object may include, for example,depiction of a physical object, such as a photograph, line drawing, orother type of depiction, or may include a brand logo, trademark, servicemark, or other design. The information about the object may furtherinclude information about the object, such as a description or otheridentifying information, information about a source of the object or ofa service represented by the object, information about rights holdersfor the image or a related product or service, etc. Additional metadatamay be embodied as a link to a product. This link may be the result of asoftware tool accessing a webpage and deriving from that webpage a linkto the hosting site. This link may provide a means by which the link isprovided along with any products within a post. This link can also beused in creating hot spot overlays on a post's video, which when clickedon by the user may result in taking the user to the brand or retailer'sproduct page to engage in the buying process. In addition to a link,other data such as in stock status, low stock status, price historyincluding price drops may be included. In operation 1815, the method mayanalyze video frames to determine a correlation between the video framesand each object. For example a correlation may be determined betweendepictions of objects in the video and the retrieved object information.Alternatively, the video file may include metadata and a comparison maybe made between the video file metadata and the object information. Forexample, the method may process each video frame with respect to eachobject in operations 1820 and 1825. In operation 1820, the method maydetermine a correlation between an object within the video frame and theobject information. The determined correlation may include a location ofan object determined to be depicted in the video frame. In operation1825, the method may extract and store coordinates of an object withinvideo frame. This operation may be performed, for example, using imagerecognition algorithms. When processing of all video frames has beencompleted, the method may continue to operation 1830. In operation 1830,the method may compare object data from video file with object data frommultiple other video files. In addition to using AI and other imagerecognition techniques to ascertain which products are associated with avideo, further comparisons may be made between unrelated videos todetermine if the same, or a subset of products, are contained. Thesemeasurements may allow for content found to have related products to beselectively presented to a consumer. In operation 1835, the method maygenerate guidance for the creator of the video file based on theanalysis and comparison of the video file. In operation 1840, the methodmay map an object location in video frames to an object URL or deep linkbased on stored object coordinates. This may include using thecoordinate data derived from the image recognition algorithm and createmetadata for the video, possibly in frame-wise fashion, to map a uniformresource locator (URL) or other link to a product, thus potentiallyallowing a user obtain further information about the object, such asoptions for purchasing, by interacting with the object within the videodisplay, such as by simply touching, clicking, or selecting the objector product shown in the video. Finally, in operation 1845, the methodmay incorporate decoration of an object in a video frame. Thesedecorations may be, for example, bounding boxes, glitter effects,highlighting effects, etc.

The disclosed methods, according to one or more embodiments, may allowpromoters, such as social media authors or influencers, to analyzeobject placement in videos and may provide guidance to enhance theperformance of such videos, thus, potentially increasing theeffectiveness of the produced videos in promoting items and marketingproducts and services.

Other embodiments of the disclosure will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A computer-implemented method for ingesting andpresenting a video with associated linked products and metadata as aunified actionable shopping experience, the method comprising: accessinga video file; receiving information about an object depicted in a frameof the video file; generating guidance for a creator of the video filebased on a correlation between the frame of the video file and theinformation about the object; and delivering the video file toconsumers.
 2. The computer-implemented method of claim 1, furthercomprising: determining the correlation between the frame of the videoand the information about the object by determining a correlationbetween the object within the video frame and the information about theobject, and extracting and storing coordinates of the object within thevideo frame.
 3. The computer-implemented method of claim 2, furthercomprising: mapping the coordinates of the object within the video frameto an external address, the external address being a link to a merchantportal or web page through which the object may be purchased.
 4. Thecomputer-implemented method of claim 3, wherein a user interaction withthe object within the video frame provides the user with furtherinformation about the object.
 5. The computer-implemented method ofclaim 3, further comprising: combining the video file and informationabout the object into a social media post, wherein delivering the videofile to the consumers comprises delivering the social media post to theconsumers.
 6. The computer-implemented method of claim 2, furthercomprising: incorporating a decoration of the object in the video frame.7. The computer-implemented method of claim 6, wherein the decoration isone of a bounding box, glitter effects, or highlighting effects.
 8. Asystem for ingesting and presenting a video with associated linkedproducts and metadata as a unified actionable shopping experience, thesystem comprising: a data storage device storing instructions in anelectronic storage medium; and a processor configured to execute theinstructions to perform operations including: accessing a video file;receiving information about an object depicted in a frame of the videofile; generating guidance for a creator of the video file based on acorrelation between the frame of the video file and the informationabout the object; and delivering the video file to consumers.
 9. Thesystem of claim 8, wherein the operations further include: determiningthe correlation between the frame of the video and the information aboutthe object by determining a correlation between the object within thevideo frame and the information about the object, and extracting andstoring coordinates of the object within the video frame.
 10. The systemof claim 9, wherein the operations further include: mapping thecoordinates of the object within the video frame to an external address,the external address being a link to a merchant portal or web pagethrough which the object may be purchased.
 11. The system of claim 10,wherein a user interaction with the object within the video frameprovides the user with further information about the object.
 12. Thesystem of claim 9, wherein the operations further include: incorporatinga decoration of the object in the video frame.
 13. The system of claim10, wherein the operations further include: combining the video file andinformation about the object into a social media post, whereindelivering the video file to the consumers comprises delivering thesocial media post to the consumers.
 14. A non-transitorymachine-readable medium storing instructions that, when executed by acomputing system, cause the computing system to perform operations foringesting and presenting a video with associated linked products andmetadata as a unified actionable shopping experience, the operationscomprising: accessing a video file; receiving information about anobject depicted in a frame of the video file; generating guidance for acreator of the video file based on a correlation between the frame ofthe video file and the information about the object; and delivering thevideo file to consumers.
 15. The non-transitory machine-readable mediumof claim 14, the operations further comprising: determining thecorrelation between the frame of the video and the information about theobject by determining a correlation between the object within the videoframe and the information about the object, and extracting and storingcoordinates of the object within the video frame.
 16. The non-transitorymachine-readable medium of claim 15, the operations further comprising:mapping the coordinates of the object within the video frame to anexternal address, the external address being a link to a merchant portalor web page through which the object may be purchased.